Decison Boundary and hyperbolic unfolding

Decision boundary of a neural network

Dataset with two classes

Train a neural network on the binary classification task

# define fully connected NN architecture
circle_detect = Net(0, [2,10,20])


# Train neural net on binary classification task
_ = train_classification_nn(circle_detect, data, label, n_epochs=15)

epoch	train_loss	valid_loss	accuracy	time
0	0.734842	0.725282	0.405000	00:00
1	0.690313	0.658216	0.366000	00:00
2	0.592282	0.542293	0.932000	00:01
3	0.476480	0.434061	0.979000	00:00
4	0.394097	0.368978	1.000000	00:00
5	0.351687	0.340134	1.000000	00:01
6	0.332897	0.327614	1.000000	00:01
7	0.323875	0.321176	1.000000	00:01
8	0.319400	0.318098	1.000000	00:01
9	0.317187	0.316501	1.000000	00:00
10	0.316008	0.315628	1.000000	00:00
11	0.315367	0.315155	1.000000	00:00
12	0.315033	0.314920	1.000000	00:00
13	0.314886	0.314832	1.000000	00:00
14	0.314844	0.314819	1.000000	00:00

Contour plot of the neural network function

The descion boundray is $\left\{x\in \mathbb R^2:\ nn(x)=\frac 12\right\}$, i.e. all points where the output of the neural network is ambiguous.

Ways to compute the decision boundary

First Approach: Sampling algorithm

• Using methods described in "On The Topological Expressive Power of Neural Networks", brought to our attention by Stefania at the last AppTop Meet-up.
• Implementation by Matteo.

Gradient flow method

• Pushing point to the decision boundary.
• Using automatic differentiation of neural networks.
• Fully implemented in Pytorch hence it can be strong acceleration via GPUs.

Sample points $x_i$ with gradients $-\nabla_{x_i} \left(nn(x_i)-\frac 12\right)^2$.

After 50 iterations and a filtration with respect to $|nn(x_i)-0.5|>0.01$ we end up with the red points:

Comparision with the previously plotted decision boundary.

Hyperbolic unfolding

• Conformal transformation the euclidian metric with factor $1/(nn-\frac 12)$.
• The resulting metric space is (Gromov-)hyperbolic with the decision boundary as boundary at infinity.
• Geodesic ray converge to the boundary at infinity.
• We use numerical methods for solving geodesic equations (second order differential equations).

Computed decision boundary with dataset

Entanglement problem for tori

Decision boundary for entangled tori

Decision boundary for unentangled tori